150 research outputs found
A Concise yet Effective model for Non-Aligned Incomplete Multi-view and Missing Multi-label Learning
In reality, learning from multi-view multi-label data inevitably confronts
three challenges: missing labels, incomplete views, and non-aligned views.
Existing methods mainly concern the first two and commonly need multiple
assumptions to attack them, making even state-of-the-arts involve at least two
explicit hyper-parameters such that model selection is quite difficult. More
roughly, they will fail in handling the third challenge, let alone addressing
the three jointly. In this paper, we aim at meeting these under the least
assumption by building a concise yet effective model with just one
hyper-parameter. To ease insufficiency of available labels, we exploit not only
the consensus of multiple views but also the global and local structures hidden
among multiple labels. Specifically, we introduce an indicator matrix to tackle
the first two challenges in a regression form while aligning the same
individual labels and all labels of different views in a common label space to
battle the third challenge. In aligning, we characterize the global and local
structures of multiple labels to be high-rank and low-rank, respectively.
Subsequently, an efficient algorithm with linear time complexity in the number
of samples is established. Finally, even without view-alignment, our method
substantially outperforms state-of-the-arts with view-alignment on five real
datasets.Comment: 15 pages, 7 figure
Near-Optimal Decentralized Momentum Method for Nonconvex-PL Minimax Problems
Minimax optimization plays an important role in many machine learning tasks
such as generative adversarial networks (GANs) and adversarial training.
Although recently a wide variety of optimization methods have been proposed to
solve the minimax problems, most of them ignore the distributed setting where
the data is distributed on multiple workers. Meanwhile, the existing
decentralized minimax optimization methods rely on the strictly assumptions
such as (strongly) concavity and variational inequality conditions. In the
paper, thus, we propose an efficient decentralized momentum-based gradient
descent ascent (DM-GDA) method for the distributed nonconvex-PL minimax
optimization, which is nonconvex in primal variable and is nonconcave in dual
variable and satisfies the Polyak-Lojasiewicz (PL) condition. In particular,
our DM-GDA method simultaneously uses the momentum-based techniques to update
variables and estimate the stochastic gradients. Moreover, we provide a solid
convergence analysis for our DM-GDA method, and prove that it obtains a
near-optimal gradient complexity of for finding an
-stationary solution of the nonconvex-PL stochastic minimax problems,
which reaches the lower bound of nonconvex stochastic optimization. To the best
of our knowledge, we first study the decentralized algorithm for Nonconvex-PL
stochastic minimax optimization over a network.Comment: 31 page
Convex Subspace Clustering by Adaptive Block Diagonal Representation
Subspace clustering is a class of extensively studied clustering methods and
the spectral-type approaches are its important subclass whose key first step is
to learn a coefficient matrix with block diagonal structure. To realize this
step, sparse subspace clustering (SSC), low rank representation (LRR) and block
diagonal representation (BDR) were successively proposed and have become the
state-of-the-arts (SOTAs). Among them, the former two minimize their convex
objectives by imposing sparsity and low rankness on the coefficient matrix
respectively, but so-desired block diagonality cannot neccesarily be guaranteed
practically while the latter designs a block diagonal matrix induced
regularizer but sacrifices convexity. For solving this dilemma, inspired by
Convex Biclustering, in this paper, we propose a simple yet efficient
spectral-type subspace clustering method named Adaptive Block Diagonal
Representation (ABDR) which strives to pursue so-desired block diagonality as
BDR by coercively fusing the columns/rows of the coefficient matrix via a
specially designed convex regularizer, consequently, ABDR naturally enjoys
their merits and can adaptively form more desired block diagonality than the
SOTAs without needing to prefix the number of blocks as done in BDR. Finally,
experimental results on synthetic and real benchmarks demonstrate the
superiority of ABDR.Comment: 13 pages, 11 figures, 8 table
- …